Method and apparatus for recognizing illegal behavior in unattended scenario

ABSTRACT

Methods, apparatuses, and devices, including computer programs encoded on computer storage media, for recognizing an unauthorized behavior are provided. One of the methods includes: obtaining, through machine vision, body movement pattern data of a user; obtaining feature data of an object; comparing the body movement pattern data with a preset body movement pattern; and determining that the user&#39;s behavior is unauthorized in response to the body movement pattern data matching the preset body movement pattern and that: a distance between the user&#39;s body and the object is within a distance threshold over a preset length of time, the attribute category of the object is a preset category, and the external surface image does not match a default external surface image of the object. The feature data may include: position data of the object based on a radio frequency identification tag on the object, attribute category data of the object, and external surface image of the object.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/796,616, filed on Feb. 20, 2020, which is a continuation applicationof International Application No. PCT/CN2018/100977, filed on Aug. 17,2018. International Application No. PCT/CN2018/100977 claims priorityand the benefit of the Chinese Patent Application No. 201711069063.2filed with China National Intellectual Property Administration (CNIPA)of the People's Republic of China on Nov. 3, 2017. The entire contentsof all of the above-recognized applications are incorporated herein byreference.

TECHNICAL FIELD

This specification relates to the field of Internet technology, and inparticular, to a method and apparatus for recognizing an unauthorizedbehavior in an unattended scene.

BACKGROUND

With the development of science and technology, more and more unattendedscenes are seen in daily life, such as unmanned supermarkets, unmannedgyms, and unmanned KTV. However, without human supervision, anunattended scene may also be subjected to some unauthorized behaviors,such as leaving without payment after eating or drinking in unmannedsupermarkets, damaging fitness facilities in unmanned gyms, etc.Therefore, an effective solution for recognizing unauthorized behaviorsneeds to be provided in unattended scenes.

SUMMARY

This specification discloses a method and apparatus for recognizing anunauthorized behavior in, for example, an unattended scene.

In one aspect, this specification provides a method for recognizing anunauthorized behavior, comprising: obtaining, through machine vision,body movement pattern data of a user; obtaining feature data of anobject; comparing the body movement pattern data with a preset bodymovement pattern; and determining that the user's behavior isunauthorized in response to the body movement pattern data matching thepreset body movement pattern and that: a distance between the user'sbody and the object is within a distance threshold over a preset lengthof time, the attribute category of the object is a preset category, andthe external surface image data does not match a default externalsurface image of the object. The feature data may include: position dataof the object based on a radio frequency identification tag on theobject, attribute category data of the object, and external surfaceimage data of the object.

In another aspect, the specification provides an apparatus forrecognizing an unauthorized behavior, comprising: a first acquisitionunit, configured to obtain body data of a user; a second acquisitionunit, configured to obtain feature data of an object; and a behaviorrecognition unit configured to determine whether the user performs anunauthorized behavior according to the body data and the feature data ofthe object.

In yet another aspect, the specification also provides a device forrecognizing an unauthorized behavior. The device may include one or moreprocessors and a non-transitory computer-readable memory coupled to theone or more processors and configured with instructions executable bythe one or more processors to perform operations. The operations mayinclude: obtaining body data of a user; obtaining feature data of anobject in the unattended scene; comparing the body data with a presetbody movement pattern; and in response to the body data matching thepreset body movement pattern, determining the unauthorized behavior ofthe user based on the body data and the feature data.

In still another aspect, the specification further provides anon-transitory computer-readable storage medium for recognizing anunauthorized behavior. The non-transitory computer-readable storagemedium may store instructions executable by one or more processors tocause the one or more processors to perform operations. The operationsmay include: obtaining body data of a user; obtaining feature data of anobject; comparing the body data with a preset body movement pattern; andin response to the body data matching the preset body movement pattern,determining the unauthorized behavior of the user based on the body dataand the feature data.

In a further aspect, the specification may also provide another methodfor recognizing an unauthorized behavior, comprising: obtaining bodydata of a user; obtaining feature data of an object in the unattendedscene; determining, according to a behavior recognition model, theunauthorized behavior of the user based on the body data and the featuredata with a preset body movement pattern. The behavior recognition modelmay be trained with results of recognizing unauthorized behaviors.

Another method for recognizing an unauthorized behavior is also providedthe specification. The method may include: obtaining, through machinevision, body data of a user; obtaining feature data of an object;comparing the body data with a preset body movement pattern; and inresponse to the body data matching the preset body movement pattern andthe feature data meeting preset conditions, determining the unauthorizedbehavior of the user based on the body data and the feature data.

In this specification, the body data of the user and the feature data ofthe object, in unattended scenes, can be obtained, and actual conditionsof the user and the object are combined to determine an unauthorizedbehavior, thereby achieving effective recognition of the unauthorizedbehavior in the unattended scene.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow chart of a method for recognizing an unauthorizedbehavior in an unattended scene according to an exemplary embodiment ofthis specification;

FIG. 2 is a flow chart of another method for recognizing an unauthorizedbehavior in an unattended scene according to an exemplary embodiment ofthis specification;

FIG. 3 is a flow chart of another method for recognizing an unauthorizedbehavior in an unattended scene according to an exemplary embodiment ofthis specification;

FIG. 4 is a schematic structural diagram of an apparatus for recognizingan unauthorized behavior according to an exemplary embodiment of thisspecification; and

FIG. 5 is a block diagram of an apparatus for recognizing anunauthorized behavior according to an exemplary embodiment of thisspecification.

DETAILED DESCRIPTION

Exemplary embodiments are described in detail herein, examples of whichare illustrated in the drawings. When the following description involvesdrawings, the same numbers in different drawings represent the same orsimilar elements unless otherwise indicated. The implementation mannersdescribed in the following exemplary embodiments do not represent allimplementation manners consistent with this specification. Conversely,they are merely examples of apparatuses and methods consistent with someaspects of this specification, as detailed in the attached claims.

Terms used in this specification are only aimed at describing specificembodiments rather than limiting this specification. In thisspecification and the claims, the singular forms “a,” “an,” and “the”are intended to indicate the plural forms as well, unless the contextclearly indicates otherwise. It should also be understood that the term“and/or” used herein refers to and includes any or all possiblecombinations of one or more associated listed items.

It should be understood that although terms such as first, second, andthird may be used in this specification to describe various information,the information should not be limited thereto. These terms are only usedto distinguish one another for information of the same type. Forexample, without departing from the scope of this specification, firstinformation may also be referred to as second information; andsimilarly, second information may also be referred to as firstinformation. Depending on the context, the word “if” used herein can beinterpreted as “at the time of” “when,” or “in response todetermination.”

FIG. 1 is a schematic flow chart of a method for recognizing anunauthorized behavior in an unattended scene according to an exemplaryembodiment of this specification.

The method for recognizing an unauthorized behavior in an unattendedscene can be applied to a front-end recognition device in an unattendedscene, such as an recognition device with a processing function deployedin an unattended scene. The recognition method may also be applied to aback-end device for recognition in an unattended scene, such as aback-end recognition device, which is not limited by this specification.

In FIG. 1, the method for recognizing an unauthorized behavior in anunattended scene may include the following steps.

In step 102, the method may include obtaining body data of a user in anunattended scene.

In some embodiments, the acquisition of the body data may be startedafter the user enters the unattended scene. The user may refer to aperson who uses the unattended scene. The body data may include bodybehavior data and body position data of the user; and the body of theuser may include: limbs, mouth, and other parts. In some embodiments,the body data of the user in the unattended scene may be obtained byusing a machine vision technology. The body data may also be obtained byusing other technologies, which is not limited by this specification.

In step 104, the method may include obtaining feature data of an objectin the unattended scene.

In one embodiment, the acquisition of the feature data may also bestarted after the user enters the unattended scene. Here, the object maybe an object in an unattended scene, such as goods in an unmannedsupermarket, a cabinet in which the goods are placed in the unmannedsupermarket, or fitness equipment in an unmanned gym.

The feature data may include: position data of the object, externalsurface image data of the object, vibration data obtained by a sensorplaced in the object, and the like. In some embodiments, the featuredata of the object in the unattended scene may be obtained by using amachine vision technology. The feature data may also be obtained byother means such as a sensor, which is not limited by thisspecification.

In step 106, the method may include determining whether the userperforms an unauthorized behavior according to the body data and thefeature data of the object.

In some embodiments, the obtained body data and the feature data may beanalyzed to determine whether the user in the unattended scene performsan unauthorized behavior, such as damaging goods.

In some embodiments, the obtained body data can be compared with apreset body movement pattern. The preset body movement pattern mayinclude a preset mouth movement pattern and/or a preset limb movementpattern. When the body data of the user is determined as matching thepreset body movement pattern, whether the user performs the unauthorizedbehavior may be further determined based on the body data of the userand the feature data of the object.

The body data of the user and the feature data of the object in theunattended scene can be obtained, and actual conditions of the user andthe object are combined to recognize an unauthorized behavior, therebyachieving effective recognition of the unauthorized behavior in theunattended scene.

Detailed description of this method will be provided below by using anunmanned supermarket as the unattended scene. The unauthorized behaviormay be unauthorized eating or drinking or damaging a cabinet.

In the unmanned supermarket scene, after a user (e.g., a customer of theunmanned supermarket) enters the unmanned supermarket, the acquisitionof body data of the user and feature data of an object may be started todetermine whether the user performs an unauthorized behavior, and can beended after the user leaves the unmanned supermarket.

There may be several methods for determining whether the user enters orleaves the unmanned supermarket. In one embodiment, when a user triggersopening of a door of the unmanned supermarket from the outside, the usercan be determined to enter the unmanned supermarket; when the usertriggers opening of a door of the unmanned supermarket from the inside,the user can be determined to leave the unmanned supermarket. In anotherembodiment, when a recognition apparatus in the unmanned supermarketrecognizes a living person, the user may be determined as having enteredthe unmanned supermarket; when the recognition apparatus in the unmannedsupermarket cannot recognize a living person, the user can be determinedas having left the unmanned supermarket.

With respect to solutions for triggering opening of the door of theunmanned supermarket and recognizing a living person by the recognitionapparatus as described above, references may be made to relatedtechnologies, which are not elaborated in detail in this specification.In addition to the above-described solutions, other methods may furtherbe used to determine whether the user enters or leaves the unmannedsupermarket, which is not limited by this specification.

I. Unauthorized Eating or Drinking as the Unauthorized Behavior.

In the unmanned supermarket scene, a user may eat or drink in thesupermarket after entering thereinto, and leaves without payment,causing economic losses to the unmanned supermarket. As shown in FIG. 2,the method for recognizing an unauthorized eating or drinking behaviormay include the following steps.

In step 202, the method may include obtaining mouth position data of theuser in the unmanned supermarket.

In this embodiment, face features of the user may be obtained first, andthe user's mouth is then recognized, and mouth position data of the useris further obtained. For example, after the mouth of the user isrecognized, the mouth position data of the user can be calculatedaccording to the relative position between the user's mouth and anacquisition apparatus. In other embodiments, the mouth position data ofthe user may alternatively be obtained by other means, which is notlimited by this embodiment.

In some embodiments, the mouth position data may include longitudinaland latitudinal data of the mouth. In some embodiments, the mouthposition data may be relative position data of the mouth in the unmannedsupermarket. For example, a spatial point in the unmanned supermarket(for example, the foot of a wall of the unmanned supermarket) can beused as an origin to establish a spatial orthogonal coordinate system,and then coordinates of the user's mouth in the spatial orthogonalcoordinate system can be used as the mouth position data, which is notlimited by this specification.

In step 204, the method may include obtaining position data of an objectin the unmanned supermarket.

In some embodiments, the object may refer to goods to be sold in theunmanned supermarket. In some embodiments, a Radio FrequencyIdentification (RFID) tag can be placed on the goods, and the positiondata of the goods can be determined by the RFID positioning technology.In some other embodiments, the position data of the goods can bedetermined by using other positioning technologies, which is not limitedby this specification.

Since there are usually plenty of goods for sale in the unmannedsupermarket, in some embodiments, an initial position of each piece ofgoods can be stored in advance. For example, the positions of the goodson a shelf can be used as the initial positions. When the position of acertain piece of goods is determined to be not the initial position, thesubsequent steps of this embodiment can be performed.

Similar to the mouth position data of the user, the object position datamay be longitudinal and latitudinal data, or relative position data ofthe object in the unmanned supermarket, which is not limited by thisembodiment.

In step 206, the method may include calculating a distance between theuser's mouth and the object according to the mouth position data and theobject position data.

In one embodiment, when the distance between the user's mouth and theobject is calculated, the mouth position data and the object positiondata obtained at the same time point are usually used to ensure theaccuracy of the above-described distance.

In step 208, the method may include determining, when a length of timethat the distance between the mouth and the object is within a distancethreshold reaches a preset length of time, that the user performs anunauthorized eating or drinking behavior.

In some embodiments, the distance threshold may be preset, such as 20 cmor 30 cm. When the distance between the user's mouth and the goods iswithin the distance threshold, the distance between the user's mouth andthe goods may be short, and the user may be smelling the goods orunauthorizedly eating or drinking the goods.

If the user wants to smell the goods, the length of time that thedistance between the user's mouth and the goods is within the distancethreshold may be shorter; and if the user is eating or drinking thegoods unauthorizedly, the length of time that the distance between theuser's mouth and the goods is within the distance threshold may belonger. Therefore, the length of time that the distance between theuser's mouth and the goods is within the distance threshold can bedetermined. When the length of time reaches a preset length of time, theuser is determined to be eating or drinking unauthorizedly. In otherwords, the user performs an unauthorized eating or drinking behavior. Insome embodiments, the preset length of time may also be set in advance,such as 5 s or 10 s.

In some embodiments, when the length of time that the distance betweenthe user's mouth and the goods is within the preset distance thresholdin an unattended scene reaches the preset length, the user can bedetermined as performing an unauthorized eating or drinking behavior,thereby achieving the recognition of the unauthorized eating or drinkingbehavior in the unattended scene.

In some other embodiments, in order to avoid misrecognition and toimprove accuracy in recognizing an unauthorized eating or drinkingbehavior, mouth behavior data of the user may also be obtained as thebody data of the user; and compared with a preset mouth movement pattern(such as a chewing pattern or a swallowing pattern).

If the obtained mouth behavior data matches the preset mouth movementpattern, and the length of time that the distance between the user'smouth and the goods is within a distance threshold reaches a presetlength, the user may be determined as eating or drinking unauthorizedly,i.e., the user performs an unauthorized eating or drinking behavior. Ifthe obtained mouth behavior data does not match the preset mouthmovement pattern, the user may be determined as not performing anunauthorized eating or drinking behavior.

Whether the mouth behavior data of the user matches the preset mouthmovement pattern may be determined before or after step 206 shown inFIG. 2, or may be determined while step 206 is performed, which is notlimited by this embodiment.

In some embodiments, in order to avoid misrecognition and to improve theaccuracy in recognizing an unauthorized eating or drinking behavior,attribute category data may further be obtained as the feature data ofthe goods. The attribute category data may refer to classification ofthe goods, such as edible and inedible. The attribute category of theobject may be a preset category. If the user eats or drinksunauthorizedly in the unmanned supermarket, the goods selected by theuser must be in the edible category. Therefore, when the length of timethat the distance between the user's mouth and the goods is within adistance threshold reaches a preset length and an attribute category ofthe goods falls in the edible category, the user can be determined aseating or drinking unauthorizedly. For example, if the attributecategory of the goods is bottled drinking water, and the length of timethat the distance between the user and the bottled drinking water iswithin a distance threshold reaches a preset length, the user can beconsidered as drinking the bottled drinking water unauthorizedly.

If the length of time that the distance between the user's mouth and thegoods is within a distance threshold reaches a preset length but theattribute category of the goods does not fall in the edible category,the user may not be determined as eating or drinking unauthorizedly. Forexample, when the attribute category of the goods is a householdcommodity and the length of time that the distance between the user andthe household commodity is within a distance threshold reaches a presetlength, the user may not be determined as performing an unauthorizedeating or drinking behavior.

The attribute category of the goods can be determined before or afterstep 206 shown in FIG. 2, or may be determined while step 206 isperformed, which is not limited by this specification.

In some embodiments, in order to avoid misrecognition and to improve theaccuracy in recognizing an unauthorized eating or drinking behavior,external surface image data of the goods may further be obtained as thefeature data of the goods. In some embodiments, the external surfaceimage data may refer to an external package image of the goods.

Some edible goods in the unmanned supermarket can only be eaten afterunpacking. Thus, a user's unauthorized eating or drinking behavior maybe recognized by determining whether the external surface image data ofthe goods matches a default external surface image. In some embodiments,the default external surface image can be stored in advance, and theexternal surface image data of the goods can be compared with thedefault external surface image. If the external surface image data ofthe goods matches the default external surface image, the externalpackage of the goods may be not damaged; and if the external surfaceimage data of the goods does not match the default external surfaceimage, the external package of the goods may be damaged. For example,when the length of time that the distance between the user's mouth andthe goods is within a distance threshold reaches a preset length and theexternal package of the goods is damaged, the user may be determined asperforming an unauthorized behavior.

The external surface image data of the goods can be matched before orafter step 206 shown in FIG. 2, or may be matched while step 206 isperformed, which is not limited by this specification.

In some embodiments, the above-described solutions may be combined, orother recognition solutions may be adopted to recognize an unauthorizedeating or drinking behavior, which is not limited by this specification.

II. Damaging a Cabinet as the Unauthorized Behavior.

In the unmanned supermarket scene, after entering the unmannedsupermarket, the user may damage the goods or other objects in theunmanned supermarket. For example, the user may kick a cabinet, causinglosses to the unmanned supermarket.

FIG. 3 illustrates an exemplary recognition method for the unauthorizedbehavior of damaging a cabinet, and the method may include the followingsteps.

In step 302, the method may include obtaining limb behavior data of theuser in the unmanned supermarket.

In this embodiment, the limb behavior data may include arm behaviordata, leg behavior data, and the like. The limb behavior data of theuser may be obtained over a period of time for analyzing the user's limbmovement.

In step 304, the method may include obtaining vibration data of thecabinet in the unmanned supermarket.

In some embodiments, a sensor such as a gyroscope sensor or anacceleration sensor may be placed on an object (e.g. a cabinet) in theunmanned supermarket to obtain the vibration data of the cabinet. Insome other embodiments, no sensors are placed on the object, and thevibration data of the cabinet may be obtained alternatively by usingother technologies such as a machine vision technology. Whether thecabinet vibrates can be determined based on the obtained vibration data.

In step 306, the method may include determining, when the limb behaviordata of the user matches a preset limb movement pattern and the cabinetvibrates, that the user performs an unauthorized behavior.

Based on step 302, after the limb behavior data of the user is obtained,the limb behavior data can be compared with a preset limb movementpattern. Whether the limb behavior data matches the preset limb movementpattern, such as a “smashing” pattern, a “kicking” pattern, or a“stepping on” pattern, can be determined. If the limb behavior datamatches the preset disruptive limb movement pattern, the user may bedamaging goods in the unmanned supermarket, and the recognition of theunauthorized behavior may be continued. If the limb behavior data doesnot match the preset limb movement pattern, the user may be determinedas not performing an unauthorized behavior of damaging goods in theunmanned supermarket.

Based on step 304, after the vibration data of the cabinet is obtained,a vibration state of the object, i.e., whether the cabinet vibrates, canbe determined according to the vibration data. If the cabinet vibratesand the limb behavior data of the user matches the above-describedpreset limb movement pattern, the user can be determined as performingan unauthorized behavior of damaging the cabinet. If the cabinet doesnot vibrate, the user may be determined as not performing anunauthorized behavior of damaging the cabinet.

When the limb behavior data of the user in the unattended scene matchesthe preset disruptive limb movement pattern and the object vibrates, theuser can be determined as performing an unauthorized behavior ofdamaging an object, thereby achieving the recognition of theunauthorized disruptive behavior in the unattended scene.

In some embodiments, during the recognition of an unauthorized damagingbehavior, the distance between the object and the user's limb thatmatches the limb movement pattern may further be calculated. If thedistance is within a preset range, for example, the distance is short,the user may be determined as performing an unauthorized behavior ofdamaging the object.

In some embodiments, the user may damage goods to be sold in theunmanned supermarket other than the object that are not for sale, e.g. acabinet. For example, the user may tear the external package of thegoods. Therefore, during the recognition of an unauthorized disruptivebehavior, external surface image data of the object may further beobtained; and whether the obtained external surface image data matches adefault external surface image of the object may be used to determinewhether the external package of the object is damaged.

External surface image data of objects that are not for sale, forexample, cabinets in the unmanned supermarket, fitness equipment in theunmanned gym and other objects, may also be obtained and matched withdefault external surface images, which is not limited by thisspecification. However, as objects such as cabinets and fitnessequipment are usually sturdy, external surface images of the objects maynot change even if the objects are damaged by the user. In order toimprove the accuracy of recognition, other data further can be obtainedfor recognition.

In some embodiments, the above-described recognition solutions can becombined to recognize whether the user performs an unauthorizeddisruptive behavior or other recognition solutions can be adopted, whichis not limited by this specification.

In some embodiments, developers of the recognition solutions mayestablish a corresponding behavior recognition model first, and thenimprove the behavior recognition model through continuous training. Insome embodiments, the unauthorized behavior of the user can bedetermined, according to the behavior recognition model, based on thebody data and the feature data. In some embodiments, the unauthorizedbehavior of the user can be determined, according to the behaviorrecognition model, with a preset body movement pattern. The behaviorrecognition model can be trained with previous results of recognizingunauthorized behaviors in unattended scenes. Before the behaviorrecognition model is perfect, if the user's unauthorized behavior cannotbe determined according to the body data of the user and the featuredata of the object, a determination request may be output to relevantpersonnel. The personnel can make a determination manually and return aresult of determination to improve the behavior recognition model. Insome embodiments, when the user's unauthorized behavior cannot bedetermined according to the body data of the user and the feature dataof the object, the acquisition and recognition of the body data andfeature data may be continued. If the unauthorized behavior still cannotbe determined after a certain period of time, a determination requestmay also be output to the relevant personnel to request a manualintervention.

After the unauthorized behavior of the user is recognized, the user maybe warned or penalized for the unauthorized behavior in different ways.

In one embodiment, when the user performs an unauthorized behavior, theuser can be subjected to a credit penalty. For example, before enteringthe unattended scene, the user can undergo identity authentication, suchas scanning a QR code before entering the unmanned supermarket to allowthe unmanned supermarket to obtain the user's account information. Ifthe user performs an unauthorized behavior, the credit information ofhis account can be updated in a negative direction. For example, acredit rating of the user is lowered.

In another embodiment, if the user performs an unauthorized behavior,the unauthorized behavior may be reported in the unattended scene. Forexample, the unauthorized behavior of the user may be reported in audioform or may be played back in video form as a warning.

In another embodiment, if the user performs an unauthorized behavior, anamount of resources corresponding to the object targeted by theunauthorized behavior may be added to the user's bill. The amount ofresources may refer to the value of the object. For example, if the userunauthorizedly eats potato chips in the unmanned supermarket, the costof the potato chips can be added to the user's bill. The user will stillbe charged for the corresponding fees even if the user discards thepackage in the unmanned supermarket after eating the potato chips. Ifthe user damages a cabinet in the unmanned supermarket, a finecorresponding to the behavior of damaging the cabinet can be added tothe user's bill to achieve the purpose of penalizing the user.

When the user performs an unauthorized behavior, the user may bepenalized in other ways (for example, the unauthorized behavior of theuser is made public in the industry), which is not limited by thisspecification.

Corresponding to the above-described embodiments of the method forrecognizing an unauthorized behavior in an unattended scene, thisspecification further provides an apparatus for recognizing anunauthorized behavior in an unattended scene.

The embodiments of the apparatus for recognizing an unauthorizedbehavior in an unattended scene in this specification can be applied toa recognition device. The apparatus embodiment may be achieved bysoftware, hardware, or a combination thereof. Using software to achievethe apparatus embodiment as an example, the apparatus may execute acorresponding computer program instruction in a non-volatile memory, bya processor in the recognition device in which the apparatus is located,into an internal storage. In terms of hardware, FIG. 4 is a hardwarestructural diagram of a recognition device in which the apparatus forrecognizing an unauthorized behavior in an unattended scene is located.In addition to a processor, an internal storage, a network interface,and a non-volatile memory shown in FIG. 4. In some embodiments, therecognition device in which the apparatus of this embodiment is locatedmay further include other hardware according to actual functions of therecognition device. Details are not described herein.

FIG. 5 is a block diagram of an apparatus for recognizing anunauthorized behavior in an unattended scene according to an exemplaryembodiment of this specification. The apparatus 400 for recognizing anunauthorized behavior in an unattended scene can be applied to theabove-described recognition device shown in FIG. 4, and includes a firstacquisition unit 401, a second acquisition unit 402, a behaviorrecognition unit 403, a credit penalty unit 404, a report penalty unit405, and a resource penalty unit 406.

The first acquisition unit 401 is configured to obtain body data of auser in an unattended scene. The second acquisition unit 402 isconfigured to obtain feature data of an object in the unattended scene.The behavior recognition unit 403 is configured to determine, accordingto the body data and the feature data of the object, whether the userperforms an unauthorized behavior.

In some embodiments, the body data includes mouth position data of theuser; and the feature data includes object position data of the object.The behavior recognition unit 403 may be configured to calculate thedistance between the user's mouth and the object according to the mouthposition data and the object position data. When the length of time thatthe distance between the mouth and the object is within a distancethreshold reaches a preset length, determines that the user performs anunauthorized behavior.

In some embodiments, the body data may further include mouth behaviordata of the user. When the length of time that the distance between themouth and the object is within a distance threshold reaches a presetlength and the mouth behavior data of the user matches a preset mouthmovement pattern, the behavior recognition unit 403 is configured todetermine that the user performs an unauthorized behavior.

In some embodiments, the feature data further includes attributecategory data of the object. When the length of time that the distancebetween the mouth and the object is within a distance threshold reachesa preset length and the attribute category of the object falls in theedible category, the behavior recognition unit 403 is configured todetermine that the user performs an unauthorized behavior.

In some embodiments, the feature data further includes external surfaceimage data of the object. When the length of time that the distancebetween the mouth and the object is within a distance threshold reachesa preset length and the external surface image of the object does notmatch a default external surface image, the behavior recognition unit403 is configured to determine that the user performs an unauthorizedbehavior.

In some embodiments, the body data may include limb data of the user;and the feature data may include vibration data of the object. When thelimb behavior data of the user matches a preset limb movement patternand the object vibrates, the behavior recognition unit 403 is configuredto determine that the user performs an unauthorized behavior. In someembodiments, the vibration data can be obtained by a sensor placed onthe object.

The credit penalty unit 404 is configured to inflict a credit penalty onthe user when the user performs an unauthorized behavior. The reportpenalty unit 405 is configured to report, when the user performs anunauthorized behavior, the unauthorized behavior in the unattendedscene. The resource penalty unit 406 is configured to add an amount ofresources corresponding to the object to the user's bill when the userperforms an unauthorized behavior.

Since the apparatus embodiments correspond to the method embodiments,for relevant portions, reference may be made to the specifications inthe method embodiments. The above-described apparatus embodiments areonly examples. Units described as separate components may or may not bephysically separated, and components displayed as units may or may notbe physical units, i.e., may be located at one place or be distributedon multiple network units. Some or all of the modules may be selectedaccording to actual needs to achieve the objectives of the solutions inthis specification. A person of ordinary skill in the art can understandand implement the solutions without creative efforts.

Systems, apparatuses, modules, or units described in the aboveembodiments may be implemented by computer chips or physical objects, orby products with certain functions. A typical type of implementationdevice is a computer, and a specific form of the computer may be apersonal computer, a laptop computer, a mobile phone, a camera phone, asmart phone, a personal digital assistant, a media player, a navigationdevice, an email sending and receiving device, a game console, a tabletcomputer, a wearable device, or a combination of any of these devices.

Corresponding to the method for recognizing an unauthorized behavior inan unattended scene, this specification further provides a device forrecognizing an unauthorized behavior in an unattended scene. The devicefor recognizing an unauthorized behavior in an unattended scene includesa processor and a memory for storing a machine-executable instruction.Here, the processor and the memory are usually connected to each otherthrough an internal bus. In some embodiments, the device may furtherinclude an external interface to enable communications with otherequipment or components.

By executing the machine-executable instruction for recognizing anunauthorized behavior in an unattended scene stored in the memory, theprocessor is configured to: obtain body data of a user in an unattendedscene; obtain feature data of an object in the unattended scene; anddetermine whether the user performs an unauthorized behavior accordingto the body data and the feature data of the object.

In some embodiments, the body data includes mouth position data of theuser; and the feature data includes object position data of the object.To determine whether the user performs an unauthorized behavioraccording to the body data and the feature data of the object, theprocessor is configured to: calculate the distance between the user'smouth and the object according to the mouth position data and the objectposition data. When the length of time that the distance between themouth and the object is within a distance threshold reaches a presetlength, the user is determined as performing an unauthorized behavior.

In some embodiments, the body data further includes mouth behavior dataof the user. To determine whether the user performs an unauthorizedbehavior according to the body data and the feature data of the object,the processor is configured to: when the length of time that thedistance between the mouth and the object is within a distance thresholdreaches a preset length and the mouth behavior data of the user matchesa preset mouth movement pattern, determine that the user performs anunauthorized behavior.

In some embodiments, the feature data further includes attributecategory data of the object. To determine whether the user performs anunauthorized behavior according to the body data and the feature data ofthe object, the processor is configured to: when the length of time thatthe distance between the mouth and the object is within a distancethreshold reaches a preset length and the attribute category of theobject falls in an edible category, determine that the user performs anunauthorized behavior.

In some embodiments, the feature data further includes external surfaceimage data of the object. To determine whether the user performs anunauthorized behavior according to the body data and the feature data ofthe object, the processor is configured to: when the length of time thatthe distance between the mouth and the object is within a distancethreshold reaches a preset length, and the external surface image of theobject does not match a default external surface image, determine thatthe user performs an unauthorized behavior.

In some embodiments, the body data includes limb behavior data of theuser; and the feature data includes vibration data of the object. Todetermine whether the user performs an unauthorized behavior accordingto the body data and the feature data of the object is recognized, theprocessor is configured to: when the limb behavior data of the usermatches a preset limb movement pattern and the object vibrates,determine that the user performs an unauthorized behavior. In someembodiments, the vibration data is obtained by a sensor placed on theobject.

In some embodiments, by executing the machine-executable instruction forrecognizing an unauthorized behavior in an unattended scene stored inthe memory, the processor is further configured to: when the userperforms an unauthorized behavior, inflict a credit penalty on the user.

In some embodiments, by executing the machine-executable instruction forrecognizing an unauthorized behavior in an unattended scene stored inthe memory, the processor is further configured to: if the user performsan unauthorized behavior, report the unauthorized behavior in theunattended scene.

In some other embodiments, by executing the machine-executableinstruction for recognizing an unauthorized behavior in an unattendedscene stored in the memory, the processor is further configured to: ifthe user performs an unauthorized behavior, add an amount of resourcescorresponding to the object to the user's bill.

Corresponding to the above-described embodiments of the method forrecognizing an unauthorized behavior in an unattended scene, thisspecification further provides a computer-readable storage medium. Thecomputer-readable storage medium stores a computer program, which isexecuted by a processor to perform the following steps: obtain body dataof a user in an unattended scene; obtain feature data of an object inthe unattended scene; and determine whether the user performs anunauthorized behavior according to the body data and the feature data ofthe object.

In some embodiments, the body data includes mouth position data of theuser; and the feature data includes object position data of the object.Determining whether the user performs an unauthorized behavior accordingto the body data and the feature data of the object includes:calculating the distance between the user's mouth and the objectaccording to the mouth position data and the object position data; andwhen the length of time that the distance between the mouth and theobject is within a distance threshold reaches a preset length,determining that the user performs an unauthorized behavior.

In some embodiments, the body data further includes mouth behavior dataof the user. Determining whether the user performs an unauthorizedbehavior according to the body data and the feature data of the objectincludes: when the length of time that the distance between the mouthand the object is within a distance threshold reaches a preset lengthand the mouth behavior data of the user matches a preset mouth movementpattern, determining that the user performs an unauthorized behavior.

In some embodiments, the feature data further includes attributecategory data of the object. Determining whether the user performs anunauthorized behavior according to the body data and the feature data ofthe object includes: when the length of time that the distance betweenthe mouth and the object is within a distance threshold reaches a presetlength and the attribute category of the object falls in an ediblecategory, determining that the user performs an unauthorized behavior.

In some embodiments, the feature data further includes external surfaceimage data of the object. Determining whether the user performs anunauthorized behavior according to the body data and the feature data ofthe object includes: when the length of time that the distance betweenthe mouth and the object is within a distance threshold reaches a presetlength, and the external surface image of the object does not match adefault external surface image, determining that the user performs anunauthorized behavior.

In some embodiments, the body data includes limb data of the user; andthe feature data includes vibration data of the object. Determiningwhether the user performs an unauthorized behavior according to the bodydata and the feature data of the object includes: when the limb behaviordata of the user matches a preset limb movement pattern and the objectvibrates, determining that the user performs an unauthorized behavior.In some embodiments, the vibration data is obtained by a sensor placedon the object.

In some embodiments, the processor may further perform: when the userperforms an unauthorized behavior, inflicting a credit penalty on theuser. In some other embodiments, the processor may further perform: ifthe user performs an unauthorized behavior, reporting the unauthorizedbehavior in the unattended scene. In still other embodiments, theprocessor may further perform: if the user performs an unauthorizedbehavior, add an amount of resources corresponding to the object to theuser's bill.

The embodiments of this specification are described above. Otherembodiments are within the scope of the attached claims. In some cases,actions or steps in the claims may be performed in a sequence differentfrom that in the embodiments and the desired result can still beachieved. In addition, the desired result can still be achieved if theprocesses described in the drawings are not necessarily performed in theillustrated particular or continuous sequence. In some implementations,multitasking and parallel processing are also feasible or may beadvantageous.

The above embodiments are only preferred embodiments of thisspecification and are not intended to limit this specification. Anymodification, equivalent replacement, or improvement made within thespirit and principles of this specification shall fall within the scopeof this specification.

1. A device for recognizing an unauthorized object damaging behavior,comprising: one or more processors; and a non-transitorycomputer-readable memory coupled to the one or more processors andconfigured with instructions executable by the one or more processors toperform operations comprising: obtaining body position data of a user;obtaining limb behavior data of the user; obtaining feature data of anobject, the feature data comprising: position data of the object, andvibration data of the object; comparing the limb behavior data with apreset limb movement pattern; determining whether the object vibratesbased on the vibration data; determining a distance between the user'sbody and the object based on the body position data of the user and theposition data of the object; and determining that behavior of the useris unauthorized in response to determining that: the distance betweenthe user's body and the object is within a distance threshold over apreset length of time, the limb behavior data matches the preset limbmovement pattern, and the object vibrates.
 2. The device according toclaim 1, wherein: the feature data further comprises an external surfaceimage of the object; and the operations further comprise: determiningthat the user's behavior is unauthorized in response to the externalsurface image not matching a default external surface image of theobject.
 3. The device according to claim 1, wherein: the feature datafurther comprises attribute category data of the object; and theoperations further comprise: determining that the behavior of the useris unauthorized in response to the attribute category of the objectbeing a preset category.
 4. The device according to claim 1, furthercomprising: in response to determining that the behavior of the user isunauthorized, adding an amount of resources corresponding to the objectto a bill of the user.
 5. The device according to claim 1, furthercomprising: in response to determining that the behavior of the user isunauthorized, lowering a credit rating of the user.
 6. The deviceaccording to claim 1, further comprising: in response to determiningthat the behavior of the user is unauthorized, reporting the behavior ofthe user.
 7. The device according to claim 1, further comprising: inresponse to determining that the behavior of the user is unauthorized,warning the user.
 8. A non-transitory computer-readable memory coupledto the one or more processors and configured with instructionsexecutable by the one or more processors to perform operationscomprising: obtaining body position data of the user; obtaining limbbehavior data of a user; obtaining feature data of an object, thefeature data comprising: position data of the object, and vibration dataof the object; comparing the limb behavior data with a preset limbmovement pattern; determining whether the object vibrates based on thevibration data; determining a distance between the user's body and theobject based on the body position data of the user and the position dataof the object; and determining that behavior of the user is unauthorizedin response to the limb behavior data matching the preset limb movementpattern and that: the distance between the user's body and the object iswithin a distance threshold over a preset length of time, the limbbehavior data matches the preset limb movement pattern, and the objectvibrates.
 9. The non-transitory computer-readable memory according toclaim 8, wherein: the feature data further comprises an external surfaceimage of the object; and the operations further comprise: determiningthat the user's behavior is unauthorized in response to the externalsurface image not matching a default external surface image of theobject.
 10. The non-transitory computer-readable memory according toclaim 8, wherein: the feature data further comprises attribute categorydata of the object; and the operations further comprise: determiningthat the behavior of the user is unauthorized in response to theattribute category of the object being a preset category.
 11. Thenon-transitory computer-readable memory according to claim 8, whereinthe operations further comprise: in response to determining that thebehavior of the user is unauthorized, adding an amount of resourcescorresponding to the object to a bill of the user.
 12. Thenon-transitory computer-readable memory according to claim 8, whereinthe operations further comprise: in response to determining that thebehavior of the user is unauthorized, lowering a credit rating of theuser.
 13. The non-transitory computer-readable memory according to claim8, wherein the operations further comprise: in response to determiningthat the behavior of the user is unauthorized, reporting the behavior ofthe user.
 14. The non-transitory computer-readable memory according toclaim 8, wherein the operations further comprise: in response todetermining that the behavior of the user is unauthorized, warning theuser.
 15. A computer-implemented method for recognizing an unauthorizedobject damaging behavior, comprising: obtaining body position data ofthe user; obtaining limb behavior data of a user; obtaining feature dataof an object, the feature data comprising: position data of the object,and vibration data of the object; comparing the limb behavior data witha preset limb movement pattern; determining whether the object vibratesbased on the vibration data; determining a distance between the user'sbody and the object based on the body position data of the user and theposition data of the object; and determining that behavior of the useris unauthorized in response to the limb behavior data matching thepreset limb movement pattern and that: the distance between the user'sbody and the object is within a distance threshold over a preset lengthof time, the limb behavior data matches the preset limb movementpattern, and the object vibrates.
 16. The computer-implemented methodaccording to claim 15, wherein: the feature data further comprises anexternal surface image of the object; and the computer-implementedmethod further comprises: determining that the user's behavior isunauthorized in response to the external surface image not matching adefault external surface image of the object.
 17. Thecomputer-implemented method according to claim 15, wherein: the featuredata further comprises attribute category data of the object; and thecomputer-implemented method further comprises: determining that thebehavior of the user is unauthorized in response to the attributecategory of the object being a preset category.
 18. Thecomputer-implemented method according to claim 15, further comprising:in response to determining that the behavior of the user isunauthorized, adding an amount of resources corresponding to the objectto a bill of the user.
 19. The computer-implemented method according toclaim 15, further comprising: in response to determining that thebehavior of the user is unauthorized, lowering a credit rating of theuser.
 20. The computer-implemented method according to claim 15, furthercomprising: in response to determining that the behavior of the user isunauthorized, reporting the behavior of the user.